<!-- HTML header for doxygen 1.8.6-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<title>OpenCV: OCR of Hand-written Data using kNN</title>
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../jquery.js"></script>
<script type="text/javascript" src="../../dynsections.js"></script>
<script type="text/javascript" src="../../tutorial-utils.js"></script>
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../search/searchdata.js"></script>
<script type="text/javascript" src="../../search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
//<![CDATA[
MathJax.Hub.Config(
{
  TeX: {
      Macros: {
          matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
          fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
          forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
          forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
          vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
          vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
          cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
          distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
          distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
          hdotsfor: ["\\dots", 1],
          mathbbm: ["\\mathbb{#1}", 1],
          bordermatrix: ["\\matrix{#1}", 1]
      }
  }
}
);
//]]>
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<!--#include virtual="/google-search.html"-->
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
  <td style="padding-left: 0.5em;">
   <div id="projectname">OpenCV
   &#160;<span id="projectnumber">4.5.2</span>
   </div>
   <div id="projectbrief">Open Source Computer Vision</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
</script>
<script type="text/javascript" src="../../menudata.js"></script>
<script type="text/javascript" src="../../menu.js"></script>
<script type="text/javascript">
$(function() {
  initMenu('../../',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="../../d6/d00/tutorial_py_root.html">OpenCV-Python Tutorials</a></li><li class="navelem"><a class="el" href="../../d6/de2/tutorial_py_table_of_contents_ml.html">Machine Learning</a></li><li class="navelem"><a class="el" href="../../d0/d72/tutorial_py_knn_index.html">K-Nearest Neighbour</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">OCR of Hand-written Data using kNN </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><h2>Goal </h2>
<p>In this chapter:</p><ul>
<li>We will use our knowledge on kNN to build a basic OCR (Optical Character Recognition) application.</li>
<li>We will try our application on Digits and Alphabets data that comes with OpenCV.</li>
</ul>
<h2>OCR of Hand-written Digits </h2>
<p>Our goal is to build an application which can read handwritten digits. For this we need some training data and some test data. OpenCV comes with an image digits.png (in the folder opencv/samples/data/) which has 5000 handwritten digits (500 for each digit). Each digit is a 20x20 image. So our first step is to split this image into 5000 different digit images. Then for each digit (20x20 image), we flatten it into a single row with 400 pixels. That is our feature set, i.e. intensity values of all pixels. It is the simplest feature set we can create. We use the first 250 samples of each digit as training data, and the other 250 samples as test data. So let's prepare them first. </p><div class="fragment"><div class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</div><div class="line"><span class="keyword">import</span> cv2 <span class="keyword">as</span> cv</div><div class="line"></div><div class="line">img = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv.imread</a>(<span class="stringliteral">&#39;digits.png&#39;</span>)</div><div class="line">gray = <a class="code" href="../../d8/d01/group__imgproc__color__conversions.html#ga397ae87e1288a81d2363b61574eb8cab">cv.cvtColor</a>(img,cv.COLOR_BGR2GRAY)</div><div class="line"></div><div class="line"><span class="comment"># Now we split the image to 5000 cells, each 20x20 size</span></div><div class="line">cells = [np.hsplit(row,100) <span class="keywordflow">for</span> row <span class="keywordflow">in</span> np.vsplit(gray,50)]</div><div class="line"></div><div class="line"><span class="comment"># Make it into a Numpy array: its size will be (50,100,20,20)</span></div><div class="line">x = np.array(cells)</div><div class="line"></div><div class="line"><span class="comment"># Now we prepare the training data and test data</span></div><div class="line">train = x[:,:50].reshape(-1,400).astype(np.float32) <span class="comment"># Size = (2500,400)</span></div><div class="line">test = x[:,50:100].reshape(-1,400).astype(np.float32) <span class="comment"># Size = (2500,400)</span></div><div class="line"></div><div class="line"><span class="comment"># Create labels for train and test data</span></div><div class="line">k = np.arange(10)</div><div class="line">train_labels = np.repeat(k,250)[:,np.newaxis]</div><div class="line">test_labels = train_labels.copy()</div><div class="line"></div><div class="line"><span class="comment"># Initiate kNN, train it on the training data, then test it with the test data with k=1</span></div><div class="line">knn = cv.ml.KNearest_create()</div><div class="line">knn.train(train, cv.ml.ROW_SAMPLE, train_labels)</div><div class="line">ret,result,neighbours,dist = knn.findNearest(test,k=5)</div><div class="line"></div><div class="line"><span class="comment"># Now we check the accuracy of classification</span></div><div class="line"><span class="comment"># For that, compare the result with test_labels and check which are wrong</span></div><div class="line">matches = result==test_labels</div><div class="line">correct = np.count_nonzero(matches)</div><div class="line">accuracy = correct*100.0/result.size</div><div class="line"><a class="code" href="../../df/d57/namespacecv_1_1dnn.html#a701210a0203f2786cbfd04b2bd56da47">print</a>( accuracy )</div></div><!-- fragment --><p> So our basic OCR app is ready. This particular example gave me an accuracy of 91%. One option to improve accuracy is to add more data for training, especially for the digits where we had more errors.</p>
<p>Instead of finding this training data every time I start the application, I better save it, so that the next time, I can directly read this data from a file and start classification. This can be done with the help of some Numpy functions like np.savetxt, np.savez, np.load, etc. Please check the NumPy docs for more details. </p><div class="fragment"><div class="line"><span class="comment"># Save the data</span></div><div class="line">np.savez(<span class="stringliteral">&#39;knn_data.npz&#39;</span>,train=train, train_labels=train_labels)</div><div class="line"></div><div class="line"><span class="comment"># Now load the data</span></div><div class="line">with np.load(<span class="stringliteral">&#39;knn_data.npz&#39;</span>) <span class="keyword">as</span> data:</div><div class="line">    <a class="code" href="../../df/d57/namespacecv_1_1dnn.html#a701210a0203f2786cbfd04b2bd56da47">print</a>( data.files )</div><div class="line">    train = data[<span class="stringliteral">&#39;train&#39;</span>]</div><div class="line">    train_labels = data[<span class="stringliteral">&#39;train_labels&#39;</span>]</div></div><!-- fragment --><p> In my system, it takes around 4.4 MB of memory. Since we are using intensity values (uint8 data) as features, it would be better to convert the data to np.uint8 first and then save it. It takes only 1.1 MB in this case. Then while loading, you can convert back into float32.</p>
<h2>OCR of the English Alphabet </h2>
<p>Next we will do the same for the English alphabet, but there is a slight change in data and feature set. Here, instead of images, OpenCV comes with a data file, letter-recognition.data in opencv/samples/cpp/ folder. If you open it, you will see 20000 lines which may, on first sight, look like garbage. Actually, in each row, the first column is a letter which is our label. The next 16 numbers following it are the different features. These features are obtained from the <a href="http://archive.ics.uci.edu/ml/">UCI Machine Learning Repository</a>. You can find the details of these features in <a href="http://archive.ics.uci.edu/ml/datasets/Letter+Recognition">this page</a>.</p>
<p>There are 20000 samples available, so we take the first 10000 as training samples and the remaining 10000 as test samples. We should change the letters to ascii characters because we can't work with letters directly. </p><div class="fragment"><div class="line"><span class="keyword">import</span> cv2 <span class="keyword">as</span> cv</div><div class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</div><div class="line"></div><div class="line"><span class="comment"># Load the data and convert the letters to numbers</span></div><div class="line">data= np.loadtxt(<span class="stringliteral">&#39;letter-recognition.data&#39;</span>, dtype= <span class="stringliteral">&#39;float32&#39;</span>, delimiter = <span class="stringliteral">&#39;,&#39;</span>,</div><div class="line">                    converters= {0: <span class="keyword">lambda</span> ch: ord(ch)-ord(<span class="stringliteral">&#39;A&#39;</span>)})</div><div class="line"></div><div class="line"><span class="comment"># Split the dataset in two, with 10000 samples each for training and test sets</span></div><div class="line">train, test = np.vsplit(data,2)</div><div class="line"></div><div class="line"><span class="comment"># Split trainData and testData into features and responses</span></div><div class="line">responses, trainData = np.hsplit(train,[1])</div><div class="line">labels, testData = np.hsplit(test,[1])</div><div class="line"></div><div class="line"><span class="comment"># Initiate the kNN, classify, measure accuracy</span></div><div class="line">knn = cv.ml.KNearest_create()</div><div class="line">knn.train(trainData, cv.ml.ROW_SAMPLE, responses)</div><div class="line">ret, result, neighbours, dist = knn.findNearest(testData, k=5)</div><div class="line"></div><div class="line">correct = np.count_nonzero(result == labels)</div><div class="line">accuracy = correct*100.0/10000</div><div class="line"><a class="code" href="../../df/d57/namespacecv_1_1dnn.html#a701210a0203f2786cbfd04b2bd56da47">print</a>( accuracy )</div></div><!-- fragment --><p> It gives me an accuracy of 93.22%. Again, if you want to increase accuracy, you can iteratively add more data.</p>
<h2>Additional Resources </h2>
<ol type="1">
<li><a href="https://en.wikipedia.org/wiki/Optical_character_recognition">Wikipedia article on Optical character recognition</a></li>
</ol>
<h2>Exercises </h2>
<ol type="1">
<li>Here we used k=5. What happens if you try other values of k? Can you find a value that maximizes accuracy (minimizes the number of errors)? </li>
</ol>
</div></div><!-- contents -->
<!-- HTML footer for doxygen 1.8.6-->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Apr 2 2021 11:36:37 for OpenCV by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
</a> 1.8.13
</small></address>
<script type="text/javascript">
//<![CDATA[
addTutorialsButtons();
//]]>
</script>
</body>
</html>
